Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Java benchmark JNA, JNI and uniffi FFI sync clients together #30

Closed
wants to merge 8 commits into from
Closed

Java benchmark JNA, JNI and uniffi FFI sync clients together #30

wants to merge 8 commits into from

Conversation

Yury-Fridlyand
Copy link

Combine clients from #25, #28 and #29.
Changes also include:

  1. Fixes in reporting and scripts to get results in the same units
  2. Minor code cleanup

Measurement results:
java-100-2023-10-27-02-12-43.json
rust-100-2023-10-27-02-12-43.json
final-2023-10-27-02-12-43.csv

language client num_of_tasks tps get NE avg get E avg set avg
java Lettuce 10 20260.19253 0.045008557 0.044911592 0.046906018
java Lettuce 100 21654.11313 0.042557469 0.04262643 0.044153663
java Lettuce 1000 21315.89349 0.043097517 0.043543729 0.045148083
java JNI FFI Sync client 10 5964.64198 0.161514698 0.164888779 0.164582466
java JNI FFI Sync client 100 5866.771863 0.163038004 0.167714576 0.168228983
java JNI FFI Sync client 1000 5831.661569 0.163259791 0.169058901 0.168691754
java JNA babushka 10 8741.583494 0.108506969 0.111920431 0.110350719
java JNA babushka 100 8836.700318 0.106958174 0.110685317 0.109737652
java JNA babushka 1000 8842.411437 0.104906635 0.110911215 0.109852961
java kotlin 10 9752.963094 0.102804235 0.09706637 0.101330983
java kotlin 100 10646.33175 0.090542028 0.090728638 0.090234131
java kotlin 1000 10949.55568 0.085714495 0.08907485 0.086585673
rust babushka 10 160256 0.061237179 0.06184574 0.064468362
rust babushka 100 468603 0.213023699 0.2131496 0.213579004
rust babushka 1000 400673 2.494716907 2.494657178 2.494737504
Full results table
language client is_cluster num_of_tasks data_size clientCount tps get_non_existing_p50_latency get_non_existing_p90_latency get_non_existing_p99_latency get_non_existing_average_latency get_non_existing_std_dev get_existing_p50_latency get_existing_p90_latency get_existing_p99_latency get_existing_average_latency get_existing_std_dev set_p50_latency set_p90_latency set_p99_latency set_average_latency set_std_dev
java Lettuce False 10 100 2 20260.19253452 0.041398 0.051579 0.08804999999999999 0.04500855741387918 0.053760795234625625 0.041361999999999996 0.051767 0.086819 0.044911591782533976 0.05042977135712108 0.042773 0.054917999999999995 0.09429599999999999 0.04690601762997095 0.05045700908970514
java Lettuce False 100 100 2 21654.11313393138 0.041402999999999995 0.046002999999999995 0.05511 0.042557468708039545 0.0427200847291482 0.041436999999999995 0.046275 0.055587 0.042626430390974406 0.0450628238360027 0.042983 0.048498 0.058115999999999994 0.04415366344514778 0.044596871288932645
java Lettuce False 1000 100 2 21315.893487690384 0.042082 0.045551999999999995 0.054583 0.04309751664724956 0.05192966175661814 0.042726 0.046449 0.055224999999999996 0.043543728915888466 0.05261605359683937 0.044184 0.048958999999999996 0.057841 0.04514808255839679 0.05196165876851208
java JNI FFI Sync client False 10 100 2 5964.641979630274 0.15417699999999998 0.182472 0.19848 0.16151469751777472 0.20024466676852962 0.160763 0.185636 0.204394 0.16488877879714578 0.16544660611556652 0.158327 0.187711 0.205869 0.1645824663135488 0.16513828419945153
java JNI FFI Sync client False 100 100 2 5866.771862548215 0.155795 0.183411 0.211055 0.16303800430819493 0.1678388110362286 0.162082 0.189412 0.221168 0.16771457596881664 0.16951553511190084 0.161354 0.18986699999999998 0.219014 0.16822898264861696 0.16897284806512378
java JNI FFI Sync client False 1000 100 2 5831.661569310142 0.158291 0.18402 0.205611 0.16325979122544818 0.1660402613408524 0.164648 0.190797 0.21306899999999998 0.16905890108789332 0.17251015641623088 0.16385 0.18964299999999998 0.212825 0.1686917538455232 0.17045767748676766
java JNA babushka False 10 100 2 8741.583493760008 0.10515899999999999 0.120197 0.146948 0.10850696909247629 0.4135553264225509 0.11098899999999999 0.127184 0.154173 0.11192043107427117 0.27789218452785236 0.11029399999999999 0.126184 0.153105 0.11035071858018844 0.11132796043621845
java JNA babushka False 100 100 2 8836.700318291581 0.104173 0.120211 0.144197 0.10695817446365653 0.3713300268223555 0.108528 0.126722 0.151386 0.11068531740977142 0.2939477169510594 0.107444 0.12390999999999999 0.14984899999999998 0.10973765182042418 0.31655476056806947
java JNA babushka False 1000 100 2 8842.411437035367 0.10221799999999999 0.119269 0.141555 0.10490663450664087 0.22234280803827633 0.10684099999999999 0.125825 0.149002 0.11091121511568756 0.25601126319320916 0.10654 0.12404599999999999 0.146724 0.10985296078234429 0.255282552366681
java kotlin False 10 100 2 9752.963094151724 0.081888 0.10113699999999999 0.13452999999999998 0.10280423466617303 1.3361693550875975 0.08304199999999999 0.102433 0.134524 0.09706637045686074 0.9916231008712902 0.08719299999999999 0.10711899999999999 0.145552 0.10133098344089247 0.7466571469642963
java kotlin False 100 100 2 10646.331753027109 0.076163 0.087778 0.10538399999999999 0.09054202796529051 1.5372679670656861 0.07755 0.08924 0.10693599999999999 0.09072863849503478 1.4734561585813806 0.078549 0.090318 0.10887 0.09023413073710011 1.396127969105792
java kotlin False 1000 100 2 10949.555679666466 0.07224 0.08151699999999999 0.099618 0.08571449539481532 1.4860475658276207 0.073651 0.08304399999999999 0.101369 0.08907484964830593 1.6105418923742088 0.075352 0.08505399999999999 0.10313499999999999 0.08658567278113187 1.392757523393188
rust babushka False 10 100 2 160256 0.055173 0.059465000000000004 0.054490000000000004 0.0612371793301679 0.013835005653682958 0.062827 0.06604499999999999 0.06755 0.06184573952259665 0.014036857452322537 0.057265 0.077334 0.074076 0.06446836209737788 0.014406847092483408
rust babushka False 100 100 2 468603 0.21901299999999999 0.20428500000000002 0.22608999999999999 0.21302369891768616 0.026310746205746242 0.191519 0.201301 0.22319899999999998 0.21314959990770418 0.025834285189758714 0.205099 0.226695 0.229657 0.21357900382950643 0.026015924317695675
rust babushka False 1000 100 2 400673 2.335035 2.4178149999999996 2.424929 2.49471690744709 0.4076044602536526 2.4312959999999997 2.379884 2.459143 2.494657177924358 0.39804972799066946 2.4143480000000004 2.50665 2.3876 2.494737504229395 0.39746784976842886

@Yury-Fridlyand
Iteration 1.
01b52fe 
Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>
@Yury-Fridlyand
JNA client - connect and get.
879710c 
Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>
@Yury-Fridlyand
Add JNI FFI sync client to benchmarking.
9324ba7 
Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>
@Yury-Fridlyand
Typo fix.
29d3cc2 
Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>
@Yury-Fridlyand
Add JNA client from #28.
f3144c0 
Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>
@Yury-Fridlyand
Add uniffi Kotlin client from #25.
4ea66f2 
Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>
@Yury-Fridlyand
Minor fixes.
e137673 
Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>
@Yury-Fridlyand
Update scripts and reporing.
4ddebdb 
Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>
@Yury-Fridlyand Yury-Fridlyand deleted the java_benchmarks-jni-jna-uniffi-together branch December 7, 2023 02:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jonathanl-bq jonathanl-bq Awaiting requested review from jonathanl-bq

@barshaul barshaul Awaiting requested review from barshaul

@shachlanAmazon shachlanAmazon Awaiting requested review from shachlanAmazon

@acarbonetto acarbonetto Awaiting requested review from acarbonetto

@Elen-Ghulam Elen-Ghulam Awaiting requested review from Elen-Ghulam

@SanHalacogluImproving SanHalacogluImproving Awaiting requested review from SanHalacogluImproving

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@Yury-Fridlyand